Voice controlled material handling mobile robotic system

ABSTRACT

An AMU system includes an Autonomous Mobile Unit (“AMU”), base station, lanyard, and Warehouse Management System (“WMS”) configured to communicate with one another over a network. The AMU includes a microphone configured to receive verbal commands from an individual. The individual can further provide verbal commands through the base station and the lanyard when worn by the individual. The lanyard can also provide a geo-fence around the individual where the AMU slows down to enhance safety.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication No. 62/832,565, filed on Apr. 11, 2019, which is herebyincorporated by reference.

BACKGROUND

With automation becoming more routine in industry, safety andoperational flexibility are becoming more of a concern. In the past,human access to robots and Autonomous Guided Vehicles (AGVs) wasseverely limited out of safety concerns. Newer sensor, computer, andsoftware technology has improved to such an extent that autonomousrobotic systems and humans can now regularly work in closer proximity toone another. With these improvements, comes an ever-growing marketdemand for more flexible, scalable, and user-friendly methods ofcontrolling and obtaining information from these systems.

Thus, there is a need for improvement in this field.

SUMMARY

This system generally concerns a unique voice controlled AutonomousMobile Unit (AMU), such as an AGV, mobile robot, and/or automatedforklift. The system is designed to enhance safety and control based onvoice command and geo-fencing information provided by a lanyard worn byan individual.

In one particular example, the AMU has a voice detection system thatincludes a microphone as well as an onboard circuit board that is ableto process both a wake word and any safety or emergency words locallywithout having to send for translation over a network so as to reducelatency. For instance, when the AMU is close to an individual, theindividual can say the word “stop”, and the AMU will stop. In anothervariation, commands can be used to give instructions to the AMU, provideinformation to the individual, and/or perform any number of safetyoperations such as stopping and turning. An individual can also providecommands to interrupt the workflow of the AMU such that the AMU willinterrupt the current workflow and perform the command requestedverbally by the individual, and once the command is satisfied, the AMUcan resume its previous workflow. The system can further include basestations that have microphones for receiving voice commands. Moreover,individuals can wear lanyards through which the individuals cancommunicate with the voice control system and control a particular AMU.For example, a supervisor can issue commands to retrieve informationand/or determine a given state of a particular device controlled by thesystem. The lanyard further can include tags that are used to locateindividuals within a facility such as a warehouse, storage lot, ormanufacturing plant. In one particular example, the AMU slows downautomatically in the vicinity of an individual wearing such a lanyardwithout even issuing a voice command.

Aspect 1 generally concerns a system that includes a autonomous mobileunit (AMU) that is responsive to voice commands.

Aspect 2 generally concerns the system of any previous aspect in whichthe voice commands include safety control commands.

Aspect 3 generally concerns the system of any previous aspect in whichthe safety control commands are configured to stop the AMU.

Aspect 4 generally concerns the system of any previous aspect in whichthe AMU has a controller to process the voice commands locally to reducelatency.

Aspect 5 generally concerns the system of any previous aspect in whichthe AMU is configured to transmit non-safety related control commandsfor remote processing.

Aspect 6 generally concerns the system of any previous aspect in whichthe controller includes a circuit board integrated with a microphone.

Aspect 7 generally concerns the system of any previous aspect in whichthe voice commands include requests for information.

Aspect 8 generally concerns the system of any previous aspect in whichthe voice commands include system control commands for controllingfunctions of the AMU.

Aspect 9 generally concerns the system of any previous aspect in whichthe system control commands control movement of the AMU.

Aspect 10 generally concerns the system of any previous aspect in whichthe AMU includes microphones for receiving the voice commands.

Aspect 11 generally concerns the system of any previous aspect in whichthe AMU includes an Automated Guided Vehicle (AGV).

Aspect 12 generally concerns the system of any previous aspect in whichthe AMU is controlled by a base station with a microphone for receivingthe voice commands.

Aspect 13 generally concerns the system of any previous aspect in whichthe AMU is controlled by a human-worn lanyard with a microphone forreceiving the voice commands.

Aspect 14 generally concerns the system of any previous aspect in whichthe lanyard includes a tracking device for location tracking.

Aspect 15 generally concerns the system of any previous aspect in whichthe AMU is configured to perform a safety action in the presence of thelanyard.

Aspect 16 generally concerns the system of any previous aspect in whichthe voice commands are configured to temporarily interrupt workflow ofthe AMU to perform a different task.

Aspect 17 generally concerns a method of operating the system of anyprevious aspect.

Further forms, objects, features, aspects, benefits, advantages, andembodiments of the present invention will become apparent from adetailed description and drawings provided herewith.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagrammatic view of an Autonomous Mobile Unit System (“AMUsystem”).

FIG. 2 is a diagrammatic view of an AMU found in the FIG. 1 AMU system.

FIG. 3 is a diagrammatic view of a base station found in the FIG. 1 AMUsystem.

FIG. 4 is a diagrammatic view of a lanyard found in the FIG. 1 AMUsystem.

FIG. 5 is a diagrammatic view of a Warehouse Management System (“WMS”)used in the FIG. 1 AMU system.

FIG. 6 is a diagrammatic view of a first example of an individualverbally controlling an automated forklift in the FIG. 1 AMU system.

FIG. 7 is a flowchart for a first technique for verbally controlling theFIG. 6 automated forklift.

FIG. 8 is a diagrammatic view of a second example of the individualverbally controlling the FIG. 6 automated forklift.

FIG. 9 is a flowchart for a second technique of verbally controlling theFIG. 6 automated forklift.

FIG. 10 is a diagrammatic view of an example of safety control of theFIG. 6 automated forklift using geo-fencing.

FIG. 11 is a flowchart for a collision avoidance technique performed bythe FIG. 6 automated forklift using geo-fencing.

DETAILED DESCRIPTION OF SELECTED EMBODIMENTS

For the purpose of promoting an understanding of the principles of theinvention, reference will now be made to the embodiments illustrated inthe drawings and specific language will be used to describe the same. Itwill nevertheless be understood that no limitation of the scope of theinvention is thereby intended. Any alterations and further modificationsin the described embodiments, and any further applications of theprinciples of the invention as described herein, are contemplated aswould normally occur to one skilled in the art to which the inventionrelates. One embodiment of the invention is shown in great detail,although it will be apparent to those skilled in the relevant art thatsome features that are not relevant to the present invention may not beshown for the sake of clarity.

The reference numerals in the following description have been organizedto aid the reader in quickly identifying the drawings where variouscomponents are first shown. In particular, the drawing in which anelement first appears is typically indicated by the left-most digit(s)in the corresponding reference number. For example, an elementidentified by a “100” series reference numeral will likely first appearin FIG. 1, an element identified by a “200” series reference numeralwill likely first appear in FIG. 2, and so on.

FIG. 1 shows a diagrammatic view of an Autonomous Mobile Unit System(“AMU system”) 100 according to one example. As shown, the AMU system100 includes at least one Autonomous Mobile Unit (“AMU”) 105, at leastone base station 110, at least one lanyard 115, and at least one WMS120. The AMU 105, base station 110, lanyard 115, and WMS 120 communicateover a network 125. The network 125 can for example include a wirelessand/or wired computer type network as well as private and/or public typenetworks. In the illustrated example, the network 125 includes awireless network, and the AMU 105, base station 110, lanyard 115, andWMS 120 communicate wirelessly.

The AMU 105 is configured to automatically or semi-automatically handleand move items, such as pallets, boxes, bags, parts, and other objects,within a storage facility like a warehouse or manufacturing plant. Inone example, the AMU 105 includes an autonomous or semi-autonomouslydriven forklift truck. In another example, the AMU 105 includes anAutomated Guided Vehicle (AGV). Looking at FIG. 2, the AMU 105 includesat least one microphone 205 that is used to voice control the AMU 105such as for safety and material handling purposes. The microphone 205 isoperatively coupled to a controller 210 that at least partiallyprocesses voice commands from the microphone 205 and controls theoperation of the AMU 105. The controller 210 has memory 215 configuredto store information.

As will be explained in greater detail below, the controller 210 isconfigured to determine whether words spoken by a human to the AMU 105via microphone 205 are related to safety issues or not. If the one ormore words (or phrases) concern safety or other urgent issues, such as“stop” or “halt”, based on data and speech recognition models stored inmemory 215, the controller 210 will automatically decode or determinethe command locally on the AMU 105 and automatically perform therequisite action (e.g., stop) without communicating with the WMS 120. Onthe other hand, if the voice command is not safety related or urgent,the controller 210 via the transceiver 235 transmits the sound file/dataas well as other data to the WMS 120 for processing. Once the voicecommand is processed by the WMS 120, the WMS 120 transmits the resultinginstructions and/or data back to the transceiver 235 of the AMU 105.

By processing these types of safety related or urgent verbal commandslocally, any lag time or latency is reduced which can be crucial tosafety. In addition, the AMU 105 is able to safely operate even if theWMS 120 and/or network 125 are down or unavailable. For instance, partsof warehouses or manufacturing plants can experience excessive radiointerference which inhibits communication with the network 125. Evenunder such conditions, the AMU 105 is able to provide additional safetycapabilities because the voice commands are processed locally on the AMU105. The controller 210 in the illustrated example is depicted as aunitary component, but in other examples, the controller 210 can beformed by multiple remotely located components that communicate with oneanother. For instance, part of the controller 210 can include acomputer, and the voice processing of the sounds from the microphone 205can be performed on a separate dedicated voice recognition circuit boardlocated proximal to the microphone 205. In this instance, the circuitboard of the controller 210 can use Automatic Speech Recognitionsoftware (“ASR software”) such as those using Hidden Markov Models(HMM), Dynamic Time Warping based approaches, and/or neural networkssuch as deep feed forward and recurrent neural networks.

The controller 210 is further operatively coupled to a Guidance,Navigation, and Control system (“GNC system”) 220 that is configured toautomatically sense the position, velocity, and/or acceleration of theAMU 105 so as to direct and control movement of the AMU 105. The AMU 105further includes one or more sensors 225 for sensing conditions (e.g.,location, objects, temperature, etc.). The sensors 225 in one exampleinclude location sensors (e.g., GPS) and a vision system, but the AMU105 can include other types of sensors 225. To interact withindividuals, the AMU 105 has at least one I/O device 230 that isoperatively coupled to the controller 210. For instance, the I/O device230 can include a display, one or more indicator lights, a speaker, asteering wheel, levers, switches, and a touch screen to just name a fewexamples. In order to communicate over the network 125, the AMU 105further has a transceiver 235 which is operatively coupled to thecontroller 210. It should be recognized that the AMU 105 further caninclude other components commonly found on AMUs 105 such as supportframes, forks, robot arms, power sources, wheels, and motors, to justname a few examples.

The base station 110 provides another way for an individual to controlthe AMU 105 via voice commands and/or receive information from the AMU105 and/or the WMS 120. Typically, but not always, the base station 110is located at a generally fixed location within a facility such as forexample near a loading dock in a warehouse. An operator or otherindividual can verbally control one or more AMUs 105 via the basestation 110. Through the base station 110, the operator can furtherprovide and/or receive information about the AMU 105, base station 110,and WMS 120 as well as other information. For instance, an individualcan view stock levels of particular items within an inventory, or thelocation and operational readiness of a particular AMU 105 in thefacility.

Referring to FIG. 3, the base station 110 includes a microphone 305configured to receive verbal or other auditory commands from theoperator. The microphone 305 is operatively coupled to a processor 310that processes commands and/or data for the base station 110. The basestation 110 further includes memory 315 to store and retrieveinformation that is communicated to the processor 310.

In the illustrated example, the processor 310 is further operativelycoupled to an I/O device 320 that receives input from and providesinformation to the operator. In one example, the I/O device 320 includesa touch display, keyboard, and speaker, but the I/O device 320 can beconfigured differently in other examples. The operator can for examplefully or partially control the operation of one or more AMUs 105 throughthe I/O device 320. The operator can also receive information, such asthe operational status of one or more AMUs 105, the base station 110,and/or the WMS 120 as well as other information, through the I/O device320. For instance, the operator can verbally request information and/orprovide commands via the microphone 305, and the processor 310 canprovide the requested information and/or acknowledgement of the commandthrough the I/O device 320. While in the illustrated embodiment, themicrophone 305 and I/O device 320 are illustrated as separatecomponents, the microphone 305 and I/O device 320 can be integratedtogether to form a unitary component.

The base station 110 further includes a transceiver 325, which isoperatively coupled to the processor 310, to allow the base station 110to communicate over the network 125 with other pieces of equipment inthe AMU system 100. The processor 310 is configured to receive voicecommands from the operator via the microphone 305 and transmit all orpart of the resulting audio file with a recording of the voice commandand/or other information to the WMS 120 via the transceiver 325. In onevariation, very little speech recognition processing of the voicecommands is performed locally by the processor 310, and instead, theaudio information of all voice commands is sent by the transceiver 325for processing by the WMS 120. Once the audio is processed by the WMS120, the resulting command and/or information is sent to the appropriateAMU 105 and/or base station 110. In the meantime, the I/O device 320 canprovide an acknowledgement of the voice command via the I/O device 320such as by providing a visual indication (e.g., by lighting an indicatorlight) and/or an audio acknowledgement (e.g., a voice stating “yourcommand has been received”).

In another variation, some voice commands are processed locally by theprocessor 310 on the base station 110 to avoid latency issues. Forexample, speech recognition of safety related commands (e.g., “stop”) inone variation are performed locally by the processor 310 of the basestation 110. The transceiver 325 is then used to directly communicatewith the equipment at issue. For example, if the operator says the words“stop all close AMUs”, the processor 310 of the base station 110 canissue a command via the transceiver 325 directly to all AMUs 105 withina predefined range (e.g., 10 meters) of the base station 110 to stop.The AMUs 105 can reply to the base station 110 to indicate that they arestopped. If a reply from a particular AMU 105 is not received, the basestation 110 can take other corrective actions such as reporting theissue to the WMS 120.

Turning to FIG. 4, the lanyard 115 provides a further way for anindividual to control the AMU 105 via voice commands. An operator orother individual can verbally control one or more AMUs 105 via thelanyard 115. Warehouses and manufacturing plants can be rather noisysuch that the microphone 205 on the AMU 105 might not be able to pick upor understand a voice command from an individual. Moreover, echoing orother poor acoustics within warehouses or other facilities can beproblematic such that a voice safety command may be unintentionallypicked up by other AMUs 105 that are remote from the individual therebyresulting in unintended actions by the remote AMUs 105. The lanyard 115is usually worn by or otherwise held in close proximity to theindividual so as to enhance sound quality and speech recognition.Moreover, as will be described below, the location of the individual canbe tracked with the lanyard 115 so that only AMUs 105 in close proximityto the individual are controlled when a safety command is issued and notremote AMUs 105 and/or those that impose no safety risk (e.g.,stationary or deactivated AMUs 105).

As shown, the lanyard 115 includes a microphone 405 configured toreceive verbal or other auditory commands from the operator. Themicrophone 405 is operatively coupled to a processor 410 that processescommands and/or data for the lanyard 115. The lanyard 115 furtherincludes memory 415 to store and retrieve information that iscommunicated to the processor 410. The lanyard 115 further includes aPosition Detection System (“PDS”) 420 that is operatively coupled to theprocessor 410. The PDS 420 is configured to determine the position ofthe lanyard 115. The PDS 420 can determine position through indoorposition determination techniques, outdoor position determinationtechniques, dead reckoning techniques, or a combination of thesetechniques. For indoor position determination, the PDS 420 can use anIndoor Positioning System (“IPS”) that locates objects or people insidea building using lights, radio waves, magnetic fields, acoustic signals,and/or other sensory information. In one particular example, the PDS 420determines position by measuring the distance to one or more anchoringnodes with known fixed positions (e.g., Wi-Fi/LiFi access points and/orBluetooth® beacons). When the AMUs 105 are for example used outdoors,such as in outdoor storage lots, the PDS 420 can utilize outdoorposition determination techniques like a satellite based technique suchas with a Global Positioning System (“GPS”). In another form, the PDS420 can include an Inertial Measurement Unit (“IMU”) that measures thespecific force of the lanyard 115, the angular rate of the lanyard 115,and sometimes the magnetic field surrounding the lanyard 115.

Some or all of the position determination can be locally determinedthrough the processor 410 and/or PDS 420 in one variation. To conserveenergy as well as reduce processing requirements, some or all of theposition determination can be offloaded to be remotely determined by theWMS 120. For instance, the lanyard 115 can transmit the raw data fromthe PDS 420 to the WMS 120, and the WMS 120 can then calculate theposition of the lanyard 115.

The lanyard 115 further includes a transceiver 425, which is operativelycoupled to the processor 410, to allow the lanyard 115 to communicateover the network 125 with other pieces of equipment in the AMU system100. The lanyard 115 further includes an Energy Storage System (“ESS”)430, such as a battery or capacitor, that provides portable power to thelanyard 115. The processor 410 is configured to receive voice commandsfrom the operator via the microphone 405 and transmit all or part of theresulting audio file with a recording of the voice command and/or otherinformation to the WMS 120 via the transceiver 425. In one variation,very little speech recognition processing of the voice commands isperformed locally by the processor 410, and instead, the audioinformation of all voice commands is sent by the transceiver 425 forprocessing by the WMS 120. Once the audio is processed by the WMS 120,the resulting command and/or information is sent to the appropriate AMU105, base station 110, and/or lanyard 115. Based on the locationinformation provided by the PDS 420, the WMS 120 can determine theappropriate AMUs 105 and/or base stations 110 proximal to the lanyard115 that should receive the operational instructions and/or information.

In another variation, some voice commands are processed locally by theprocessor 410 on the lanyard 115 to avoid latency issues. For example,speech recognition of safety related commands (e.g., “stop”) in onevariation is performed locally by the processor 410 of the lanyard 115.The transceiver 425 is then used to directly communicate with theequipment at issue. For example, if the operator says the words “stopall close AMUs”, the processor 410 of the lanyard 115 can issue acommand via the transceiver 425 directly to all AMUs 105 within apredefined range (e.g., 10 meters) of the lanyard 115 to stop based onthe location information from the PDS 420. The AMUs 105 can reply to thelanyard 115 to indicate that they are stopped. If a reply from aparticular AMU 105 is not received, the lanyard 115 can take othercorrective actions such as reporting the issue to the WMS 120.

In the illustrated example, the lanyard 115 defines a strap slot 435through which a strap 440 is looped. The strap 440 in one form can beworn around the neck of an individual, but the lanyard 115 can be wornin other ways. The strap 440 can be secured to the lanyard 115 in otherways besides through the strap slot 435. For instance, a clip can beused to secure the strap 440 to the rest of the lanyard 115. Again, thelanyard 115 is typically worn by or associated with one or moreindividuals so that the WMS 120 can track the position of individualswithin a facility, and if needed, change the movement and/or operationof AMUs 105 within the facility to avoid collisions with or otheraccidents between the AMU 105 and the individual wearing the lanyard115. In one form, the lanyard 115 is in the form of a plastic encasedcard with the strap 440 that is worn around the neck of an individual.In another example, the lanyard 115 is sized and shaped in a fashionsimilar to a credit card so that the lanyard 115 can be temporarilystored in a purse or pocket. In still yet another version, the lanyard115 can be clipped to or otherwise incorporated into clothing such as asafety vest.

Referring now to FIG. 5, the WMS 120 in one version is in the form of acomputer, such as a server, that includes software for managinginventories within a storage facility. The WMS 120 includes a networkinterface 505 that allows the WMS 120 to communicate over the network125. In one form, the network interface 505 includes a Network InterfaceCard (“NIC”). Like most computers, the WMS 120 further includes aprocessor 510, memory 515, and an I/O device 520. The memory 515 caninclude a database that tracks the location equipment as well asindividuals and supply levels of various items in the AMU system 100.The processor 510 can be used to perform speech recognition for thevoice commands when needed and provide appropriate instructions andinformation to the AMUs 105 and base stations 110 in the AMU system 100.

A technique for verbally controlling the AMU 105 in the AMU system 100,such as with a safety command, will now be described with reference toFIGS. 6 and 7. Looking at FIG. 6, the AMU 105 in the illustrated exampleis an automated forklift 605 that includes the same components as thoseshown with the FIG. 2 AMU 105. The microphones 205 for the automatedforklift 605 are only shown in FIG. 6 for the sake of clarity, but itshould be recognized that the automated forklift 605 incorporates thepreviously described components. For example, the automated forklift 605includes the controller 210, memory 215, GNC system 220, sensors 225,I/O device 230, and transceiver 235 of the types described before. Inaddition, the automated forklift 605 includes material handlingequipment in the form of one or more forks 610. In the depicted example,the automated forklift 605 is moving in the direction as indicated byarrow 615. An individual 620 issues a verbal instruction 625 which isreceived via the microphones 205 on the automated forklift 605. Whilethis technique will be described with respect to receiving andprocessing the verbal instruction 625 with the AMU 105, this techniquecan be used with the base station 110 and lanyard 115 of the AMU system100.

A flowchart 700 illustrating this technique for verbally controlling theAMU 105 (e.g., the automated forklift 605) is shown in FIG. 7. In stage705, the controller 210 of the automated forklift 605 via the microphone205 monitors for one or more verbal instructions 625. In one example,the automated forklift 605 requires a wake word (e.g., “Hey, forklift”)in order to take action on the verbal instruction 625. In otherexamples, such as when a verbal safety command is used (e.g., “stop”,“turn”, “reverse”, etc.), the wake word can be optional or not used. Asshould be recognized, when under the pressure of a hazardous situation,the individual 620 may forget the wake word, and by not requiring thewake word for a safety command, the chances of accident avoidancebetween the automated forklift 605 and individual 620 are enhanced. Inone specific example, wake words are not required in stage 705 forverbal instructions 625 that are safety related commands, and wake wordsare required for non-safety related commands, instructions, or requestsso as to reduce accidental activation.

In stage 710, the controller 210 of the automated forklift 605determines whether the verbal instruction 625 heard via the microphones205 is a safety related command. The automated forklift 605 stores inmemory 215 a list of commands or speech characteristics that areindicative of a safety related command. The safety command generallyconcerns a request for an action to be performed by the AMU 105 thatprevents imminent harm or damage to a human, a piece of equipment,and/or a structure, or even to the AMU 105. In the example illustratedin FIG. 6, the automated forklift 605 is travelling in the directionindicated by arrow 615 towards the individual 620. In this case, theverbal instruction 625 is a safety command (i.e., “Stop!”) because thecommand is intended to avoid imminent harm caused by the automatedforklift 605 running into the individual 620. It should be made clearthat this technique is intended to act as a supplement to otherpreexisting safety equipment and software on the automated forklift 605(e.g., light curtains, vision systems, safety sensors, etc.) and is notintended to replace these already existing safety features. Thecontroller 210 of the automated forklift 605 in stage 710 performs aninitial pass using local voice recognition processing to determine ifthe verbal instruction 625 is possibly a safety related command. In oneexample, probability or confidence thresholds are used to make thedetermination. To err on the side of caution, a low confidence thresholdis used (e.g., greater than 30% confidence), but higher thresholds(e.g., greater than 50%) can be used in other examples.

Once the controller 210 of the automated forklift 605 determines thatthe verbal instruction 625 is more likely than not a safety commandaccording to the thresholds stored in memory 215, the controller 210further processes the verbal instruction 625 using additional speechrecognition techniques to determine the exact safety command. Usually,but not always, safety commands are a short word or phrase of one or afew syllables. Due to the short length and/or tone of the verbalinstruction 625, the automated forklift 605 may infer that the verbalinstruction 625 concerns a safety command but may not know whichparticular safety command the verbal instruction 625 concerns. Byperforming speech recognition locally on the controller 210 of theautomated forklift 605 in stage 715, latency can be reduced which inturn speeds up response time by the automated forklift 605. Based on theidentified safety command, the controller 210 retrieves the one or morecorresponding action instructions from memory 215, and the controller210 instructs the appropriate equipment within the automated forklift605 to perform the required actions in stage 720. For example, theautomated forklift 605 in FIG. received “Stop!” as the verbalinstruction 625. Based on this safety command, the controller 210instructs the GNC system 220, braking system, and/or drivetrain to stopthe automated forklift 605. Of course, the automated forklift 605 cantake different actions depending on the safety instruction. For example,the automated forklift 605 can turn if instructed to turn or slow downif the verbal instruction 625 was to slow down. With this technique,when the base station 110 or lanyard 115 receives the verbal instruction625, the base station 110 or lanyard 115 processes the verbalinstruction 625 in the same fashion as described above. At stage 720,the base station 110 or lanyard 115 transmits the safety commanddirectly to the automated forklift 605 via network 125, and theautomated forklift 605 performs the instructed safety action (e.g.,stops).

On the other hand, if the controller 210 of the automated forklift 605determines the verbal instruction 625 is likely not a safety relatedcommand in stage 710 (or later during subsequent speech recognitionprocessing in stage 715), the controller 210 transmits a recording fileof the verbal instruction 625 and/or other data (e.g., the likelycommand based on local processing) to the WMS 120 via the transceiver235 in stage 725. At stage 730, the controller 210 of the automatedforklift 605 waits for a response from the WMS 120. After the responseis received from the WMS 120 over the network 125, the automatedforklift 605 performs the one or more actions and/or provides therequested information in stage 720. For example, the individual 620 viathe verbal instruction 625 can ask for the nearest open storagelocation, and the automated forklift 605 can identify the location withthe I/O device 230, or even travel to the open storage location based onthe instructions provided by the WMS 120. In certain cases, such as whenthe verbal instruction 625 concerns the operational status of theautomated forklift 605 (e.g., battery level), the automated forklift 605can process the verbal instruction 625 locally as well.

A technique for verbally changing the workflow or tasks of the AMU 105in the middle of the workflow will now be described with reference toFIGS. 8 and 9. The technique will be described with reference to theenvironment or conditions shown in FIG. 8, but it should be recognizedthat this technique can be used in other situations. Moreover, thetechnique will be described with reference to the FIG. 6 automatedforklift 605, but other types of AMUs 105 can be used.

As shown in FIG. 8, a first storage location 805 is where a first pallet810 is initially stored, and a second storage location 815 is where asecond pallet 820 is stored. Similarly, a third storage location 825 iswhere a third pallet 830 is stored. With this illustrated example, theautomated forklift 605 is initially instructed to travel along aninitial path 835 to the third storage location 825 in order to pick upthe third pallet 830. However, in this example, the individual 620 hasprovided one or more verbal instructions 625 to the automated forklift605 via the base station 110 to perform a different task. In this case,the individual 620 has instructed the automated forklift 605 to pick upthe second pallet 820 at the second storage location 815 and move thesecond pallet 820 to the first storage location 805. The verbalinstructions 625 for this technique are processed in the same fashion asdescribed before with respect to FIG. 6 and the flowchart 700 in FIG. 7.In the illustrated example, the verbal instruction 625 is received bythe base station 110, but in other examples, the lanyard 115 and/orautomated forklift 605 can receive and process the verbal instructions625. Given the verbal instruction 625 is a non-safety related command inthe illustrated example, the verbal instruction 625 is transmitted tothe WMS 120 (see stage 725 in FIG. 7), and the WMS 120 providesinstructions to the automated forklift 605 to travel along a detour path840 to the second storage location 815. In accordance with theinstructions, the automated forklift 605 picks up the second pallet 820with the forks 610 and moves the second pallet 820 to the first storagelocation 805. After the task of moving the second pallet 820 to thefirst storage location 805 is complete, the automated forklift 605resumes the initial task of picking up the third pallet 830 by movingalong the initial path 835 or a similar path.

A more generalized form of this technique is illustrated with aflowchart 900 in FIG. 9. In stage 905, the automated forklift 605 isperforming a first task. For the FIG. 8 example, the first task of stage905 was moving along the initial path 835 to pick up the third pallet830 at the third storage location 825. The first pallet 810 receives theverbal instruction 625 and transmits the verbal instruction 625 to theWMS 120 where the verbal instruction 625 is processed. While theautomated forklift 605 performs the first task, the automated forklift605 monitors to see if a second task is received from the WMS 120 ordirectly from the individual 620 in stage 910. If no new command isreceived, the automated forklift 605 continues to perform the first taskof stage 905 by travelling along the initial path 835. However, once theautomated forklift 605 receives the second task from the WMS 120 instage 910, the automated forklift 605 performs the second task in stage915. Once more, in the FIG. 8 example, the automated forklift 605 movesoff the initial path 835 and travels along the detour path 840 to thesecond storage location 815 where the automated forklift 605 performsthe task of moving the second pallet 820 to the first storage location805. For instance, the controller 210 of the automated forklift 605provides the path coordinates of the detour path 840 to the GNC system220. Once the second task is complete in stage 920, the automatedforklift 605 resumes performance of the first task in stage 905. For theFIG. 8 example, the controller 210 instructs the GNC system 220 to movethe automated forklift 605 back along the initial path 835 and to thethird storage location 825 in order to pick up the third pallet 830. Inother examples, subsequent verbal instructions 625 to perform additionaltasks (e.g., for third and fourth tasks) can temporarily interrupt theprior first and second tasks in a similar fashion as described before.

A safety technique for changing the operation of the AMU 105 based onthe proximity of the individual 620 will now be described with referenceto t FIGS. 10 and 11. This technique in one form is used in conjunctionwith the earlier described voice control techniques of FIGS. 6, 7, 8,and 9. This technique will be again described with reference to theautomated forklift 605 but other types of AMUs 105 can perform thistechnique. As shown in FIG. 10, the individual 620 wears the lanyard115. Once more, the lanyard 115 has the PDS 420 that provides thelocation of the individual 620 to the automated forklift 605. Thelocation of the lanyard 115 can be directly provided by the lanyard 115communicating the location information to the automated forklift 605over the network 125. Alternatively or additionally, this locationinformation for the lanyard 115 can be indirectly sent to the automatedforklift 605 via the WMS 120. The lanyard 115 sends the locationinformation (e.g., positional coordinates, velocity, acceleration, etc.)to the WMS 120 and/or the WMS 120 determines the location informationabout the lanyard 115 based on raw data from the PDS 420. The WMS 120can broadcast this location information about the lanyard 115 to allAMUs 105 in the AMU system 100 or only to select AMUs 105 that areactive and/or near the individual 620 wearing the lanyard 115. Thislocation information can be sent in a periodic manner and/or pushed outwhen the location of the lanyard 115 changes.

In one variation, the WMS 120 further provides coordinates for one ormore geo-fence safety zones around the individual 620 wearing thelanyard 115. Alternatively or additionally, the automated forklift 605determines the geo-fence safety zones. In the illustrated example, thesegeo-fence zones include an inner safety zone 1005 and an outer safetyzone 1010, but other examples can include more or less zones than isshown. Moreover, the shape of the inner safety zone 1005 and outersafety zone 1010 can be different from the rectangular grid shown inFIG. 10. For instance, the zones can have a circular/spherical or anirregular shape. FIG. 10 further show arrows that symbolize theoperational mode of the automated forklift 605 in and around thesezones. For instance, arrow 1015 shows the state of operation when theautomated forklift 605 is outside of the outer safety zone 1010, andarrow 1020 along with arrow 1025 show the operational state of theautomated forklift 605 when within the outer safety zone 1010 and inclose proximity to the inner safety zone 1005.

FIG. 11 show a flowchart 1100 for this technique. In stage 1105, theautomated forklift 605 travels at the normal or usual operational speedand direction for the automated forklift 605. The determination ofrelative location of the lanyard 115 and automated forklift 605 will bedescribed as the automated forklift 605 making this determination, butin other variations, the WMS 120 can make this zone locationdetermination either alone or in cooperation with the automated forklift605 as well as the lanyard 115. The automated forklift 605 in stage 1110determines whether or not the automated forklift 605 is in the outersafety zone 1010 for the lanyard 115. When outside the outer safety zone1010, the automated forklift 605 operates in the normal manner of stage1105 as is indicated by stage 1015. However, when the automated forklift605 enters the outer safety zone 1010, the automated forklift 605performs a first safety action in stage 1115 to avoid injuring theindividual 620. In the FIG. 10 example, when entering the outer safetyzone 1010, the automated forklift 605 slows down to a speed and/or movesin a direction as indicated by arrow 1020 where the automated forklift605 and/or individual 620 can easily recognize and make movementcorrections without colliding with one another. In other examples, theautomated forklift 605 can take alternative or additional safetyactions, such as retracting or enclosing hazardous equipment, whenentering the outer safety zone 1010. The controller 210 in stage 1120then determines if the automated forklift 605 is within or near theinner safety zone 1005 in stage 1120. When outside or not near the innersafety zone 1005, the controller 210 of the automated forklift 605 thenmonitors whether or not the automated forklift 605 is still within theouter safety zone 1010 in stage 1110. If not, the automated forklift 605returns to normal operation (1015) in stage 1105 and proceeds in thesame fashion as described before.

When the automated forklift 605 closely approaches or is inside theinner safety zone 1005 in stage 1120, the automated forklift 605performs a second safety action in stage 1125 that is typically morecautious or drastic than the first safety action. For instance, asindicated by arrow 1025 in FIG. 10, the automated forklift 605 stops orturns to avoid the individual 620 when approaching the inner safety zone1005. The automated forklift 605 can take other actions in stage 1125.For example, the automated forklift 605 can power down, move away fromthe lanyard 115 to maintain a safe distance, and/or return to a safestorage location for the automated forklift 605, to name just a fewexamples. As shown by the flowchart 1100 in FIG. 11, the controller 210continues to monitor the position of the automated forklift 605 relativeto the inner safety zone 1005 and outer safety zone 1010, and theautomated forklift 605 continues to take the appropriate safety action.Once outside the outer safety zone 1010, the forklift 605 returns tonormal operation (see arrow 1015 in FIG. 10) in stage 1105.

Glossary of Terms

The language used in the claims and specification is to only have itsplain and ordinary meaning, except as explicitly defined below. Thewords in these definitions are to only have their plain and ordinarymeaning. Such plain and ordinary meaning is inclusive of all consistentdictionary definitions from the most recently published Webster'sdictionaries and Random House dictionaries. As used in the specificationand claims, the following definitions apply to these terms and commonvariations thereof identified below.

“Autonomous Mobile Unit System” or “AMU System” generally refers to amechanism used to transport items via one or more AMUs that move alongan AMU frame. The AMUs in the AMU system are able to at least move intwo spatial directions (i.e., in a vertical direction and a horizontaldirection) along the AMU frame. In another form, the AMU is able to movein all three spatial dimensions within the AMU frame. The AMU system caninclude an infeed AMU system that typically (but not always) suppliesitems to a buffering system. The AMU system can further include adischarge AMU system that typically (but not always) discharges itemsfrom the buffering system.

“Autonomous Mobile Unit” or “AMU” generally refer to a mobile robot thatis able to automatically self-navigate between various locations. Forexample, AMUs are typically, but not always, able to automaticallynavigate by following markers, such as wires or magnets embedded in thefloor, by using lasers, and/or by using one or more vision systems. AMUsare also typically, but not always, designed to automatically avoidcollisions, such as with other AMUs, equipment, and personnel. AMUs arecommonly, but not always, used in industrial applications to movematerials around a manufacturing facility or warehouse.

“Beacon” or “Beacon Transmitter” generally refers to a system orapparatus configured to transmit data using electromagnetic energy. Thebroadcasted data may include any suitable data such as a string ofalphanumeric characters uniquely identifying one beacon from others inthe environment. Data may appear in a single field in a datagram, or inmultiple separate fields. Any suitable protocol may be used to createand transmit the datagrams using any suitable arrangement of fields. Thefields may include predetermined numbers of bits according toproprietary or commercially available protocols. One example of acommercially available protocol is the BLUETOOTH® LE (Low Energy)protocol, also referred to as BLUETOOTH® Smart protocol.

Datagrams may include one or more fields that may include a preamble,one or more header fields, an access address field, a CyclicalRedundancy Check (CRC) field, a Protocol Data Unit (PDU) field, a MediaAccess Control (MAC) address field, and a data field. The data field mayinclude a prefix and a proximity Universal Unique Identifier (UUID)which may be configured to distinguish beacons used by one organizationfrom those of another organization. Other data fields may include amajor field which may be used to identify multiple beacons as a group, aminor field which may uniquely identify a specific beacon within agroup, and a transmission power field which may indicate how far abeacon is from a receiver. The transmitter power field may include oneof a set of data values representing distance ranges such as“immediate”, “far”, or “out of range”. A transmission power field mayalso include more detailed ranging data such as the Received SignalStrength Indication (RSSI) of the beacon at a predetermined range suchas 1 meter away. This value may be compared to a current RSSI measuredby a receiver and used to calculate an approximate range.

A beacon may include a receiver allowing the beacon to beginbroadcasting after receiving a signal from another transmitter. In oneexample, a beacon may collect energy from the electromagnetic energydirected toward it and may use this energy to transmit its data inresponse. This type of “passive” beacon may only transmit when energizedto do so by some other transmitter. In another example, beacons may havea local power source such as a battery and may transmit continuouslyand/or at predetermined intervals. In either case, the data sent by thebeacon may pass through walls or other objects between the beacon and areceiver making it unnecessary to maintain an unobstructed line of sightbetween the two.

A beacon may transmit on any suitable frequency or group of frequenciesin the electromagnetic spectrum. For example, a beacon may transmit inthe Very High Frequency range (VHF), the Ultra High Frequency range(UHF), or in the Super High Frequency range (SHF). Transmissions from abeacon may be directed along a narrow beam by a directional antennasystem used by the beacon, or the beacon may use an omnidirectionalantenna system configured to broadcast the data in all directions atabout the same time. In one form, the beacon is an off-the-shelf productthat is purchased.

The data may be programmed in a memory such as a nonvolatile memory inthe beacon for repeated transmission at predetermined intervals. Forexample, transmissions may be repeated up to about every 500 ms, up toabout every 2 seconds, up to about every 30 seconds, or at intervalsgreater than 30 seconds apart. Beacons may transmit at a very lowTransmitter Power Output (TPO) and/or Effective Radiated Power (ERP).TPO or ERP may be less than about 100 milliwatts, less than about 10milliwatts, or less than about 1 milliwatts.

Examples of commercially available suitable beacon transmitters includebeacons available from Estimote, Inc. of New York, N.Y., USA, or fromGimbal, Inc., of San Diego, Calif., USA.

“Computer” generally refers to any computing device configured tocompute a result from any number of input values or variables. Acomputer may include a processor for performing calculations to processinput or output. A computer may include a memory for storing values tobe processed by the processor, or for storing the results of previousprocessing.

A computer may also be configured to accept input and output from a widearray of input and output devices for receiving or sending values. Suchdevices include other computers, keyboards, mice, visual displays,printers, industrial equipment, and systems or machinery of all typesand sizes. For example, a computer can control a network interface toperform various network communications upon request. The networkinterface may be part of the computer, or characterized as separate andremote from the computer.

A computer may be a single, physical, computing device such as a desktopcomputer, a laptop computer, or may be composed of multiple devices ofthe same type such as a group of servers operating as one device in anetworked cluster, or a heterogeneous combination of different computingdevices operating as one computer and linked together by a communicationnetwork. The communication network connected to the computer may also beconnected to a wider network such as the Internet. Thus, a computer mayinclude one or more physical processors or other computing devices orcircuitry, and may also include any suitable type of memory.

A computer may also be a virtual computing platform having an unknown orfluctuating number of physical processors and memories or memorydevices. A computer may thus be physically located in one geographicallocation or physically spread across several widely scattered locationswith multiple processors linked together by a communication network tooperate as a single computer.

The concept of “computer” and “processor” within a computer or computingdevice also encompasses any such processor or computing device servingto make calculations or comparisons as part of disclosed system.Processing operations related to threshold comparisons, rulescomparisons, calculations, and the like occurring in a computer mayoccur, for example, on separate servers, the same server with separateprocessors, or on a virtual computing environment having an unknownnumber of physical processors as described above.

A computer may be optionally coupled to one or more visual displaysand/or may include an integrated visual display. Likewise, displays maybe of the same type, or a heterogeneous combination of different visualdevices. A computer may also include one or more operator input devicessuch as a keyboard, mouse, touch screen, laser or infrared pointingdevice, or gyroscopic pointing device to name just a few representativeexamples. Also, besides a display, one or more other output devices maybe included such as a printer, plotter, industrial manufacturingmachine, 3D printer, and the like. As such, various display, input andoutput device arrangements are possible.

Multiple computers or computing devices may be configured to communicatewith one another or with other devices over wired or wirelesscommunication links to form a communication network. Networkcommunications may pass through various computers operating as networkappliances such as switches, routers, firewalls or other network devicesor interfaces before passing over other larger computer networks such asthe internet. Communications can also be passed over the communicationnetwork as wireless data transmissions carried over electromagneticwaves through transmission lines or free space. Such communicationsinclude using WiFi or other Wireless Local Area Network (WLAN) or acellular transmitter/receiver to transfer data. Such signals conform toany of a number of wireless or mobile telecommunications technologystandards such as 802.11a/b/g/n, 3G, 4G, and the like.

“Controller” generally refers to a device, using mechanical, hydraulic,pneumatic electronic techniques, and/or a microprocessor or computer,which monitors and physically alters the operating conditions of a givendynamical system. In one nonlimiting example, the controller can includean Allen Bradley brand Programmable Logic Controller (PLC). A controllermay include a processor for performing calculations to process input oroutput. A controller may include a memory for storing values to beprocessed by the processor or for storing the results of previousprocessing. A controller may also be configured to accept input andoutput from a wide array of input and output devices for receiving orsending values. Such devices include other computers, keyboards, mice,visual displays, printers, industrial equipment, and systems ormachinery of all types and sizes. For example, a controller can controla network or network interface to perform various network communicationsupon request. The network interface may be part of the controller, orcharacterized as separate and remote from the controller. A controllermay be a single, physical, computing device such as a desktop computeror a laptop computer, or may be composed of multiple devices of the sametype such as a group of servers operating as one device in a networkedcluster, or a heterogeneous combination of different computing devicesoperating as one controller and linked together by a communicationnetwork. The communication network connected to the controller may alsobe connected to a wider network such as the Internet. Thus a controllermay include one or more physical processors or other computing devicesor circuitry and may also include any suitable type of memory. Acontroller may also be a virtual computing platform having an unknown orfluctuating number of physical processors and memories or memorydevices. A controller may thus be physically located in one geographicallocation or physically spread across several widely scattered locationswith multiple processors linked together by a communication network tooperate as a single controller. Multiple controllers or computingdevices may be configured to communicate with one another or with otherdevices over wired or wireless communication links to form a network.Network communications may pass through various controllers operating asnetwork appliances such as switches, routers, firewalls or other networkdevices or interfaces before passing over other larger computer networkssuch as the Internet. Communications can also be passed over the networkas wireless data transmissions carried over electromagnetic wavesthrough transmission lines or free space. Such communications includeusing WiFi or other Wireless Local Area Network (WLAN) or a cellulartransmitter/receiver to transfer data.

“Data” generally refers to one or more values of qualitative orquantitative variables that are usually the result of measurements. Datamay be considered “atomic” as being finite individual units of specificinformation. Data can also be thought of as a value or set of valuesthat includes a frame of reference indicating some meaning associatedwith the values. For example, the number “2” alone is a symbol thatabsent some context is meaningless. The number “2” may be considered“data” when it is understood to indicate, for example, the number ofitems produced in an hour.

Data may be organized and represented in a structured format. Examplesinclude a tabular representation using rows and columns, a treerepresentation with a set of nodes considered to have a parent-childrenrelationship, or a graph representation as a set of connected nodes toname a few.

The term “data” can refer to unprocessed data or “raw data” such as acollection of numbers, characters, or other symbols representingindividual facts or opinions. Data may be collected by sensors incontrolled or uncontrolled environments, or generated by observation,recording, or by processing of other data. The word “data” may be usedin a plural or singular form. The older plural form “datum” may be usedas well.

“Energy Storage System” (ESS) or “Energy Storage Unit” generally refersto a device that captures energy produced at one time for use at a latertime. The energy can be supplied to the ESS in one or more forms, forexample including radiation, chemical, gravitational potential,electrical potential, electricity, elevated temperature, latent heat,and kinetic types of energy. The ESS converts the energy from forms thatare difficult to store to more conveniently and/or economically storableforms. By way of non-limiting examples, techniques for accumulating theenergy in the ESS can include: mechanical capturing techniques, such ascompressed air storage, flywheels, gravitational potential energydevices, springs, and hydraulic accumulators; electrical and/orelectromagnetic capturing techniques, such as using capacitors, supercapacitors, and superconducting magnetic energy storage coils;biological techniques, such as using glycogen, biofuel, and starchstorage mediums; electrochemical capturing techniques, such as usingflow batteries, rechargeable batteries, and ultra batteries; thermalcapture techniques, such as using eutectic systems, molten salt storage,phase-change materials, and steam accumulators; and/or chemical capturetechniques, such as using hydrated salts, hydrogen, and hydrogenperoxide. Common ESS examples include lithium-ion batteries and supercapacitors.

“Geo-fence” generally refers to a virtual boundary generated for a realgeographical area. The virtual boundary defined by a geo-fence may bemonitored using a positioning system and/or any other form oflocation-based service.

“Guidance, Navigation, and Control (GNC) System” generally refers to aphysical device, a virtual device, and/or a group of devices configuredto control the movement of vehicles, such as automobiles, automatedguided vehicles, ships, aircraft, drones, spacecraft, and/or othermoving objects. GNC systems are typically configured to determine adesired path of travel or trajectory of the vehicle from the vehicle'scurrent location to a designated target, as well as desired changes invelocity, rotation, and/or acceleration for following the path. The GNCsystem can include and/or communicate with sensors like compasses, GPSreceivers, Loran-C, star trackers, inertial measurement units,altimeters, environmental sensors, and the like. At a given time, suchas when the vehicle is travelling, the GNC system is configured todetermine the location (in one, two, or three dimensions) and velocityof the vehicle. For example, the GNC system is able to calculate changesin position, velocity, attitude, and/or rotation rates of a movingvehicle required to follow a certain trajectory and/or attitude profilebased on information about the state of motion of the vehicle. The GNCsystem is able to maintain or change movement of the vehicle bymanipulating forces by way of vehicle actuators, such as steeringmechanisms, thrusters, flaps, etc., to guide the vehicle whilemaintaining vehicle stability. GNC systems can be found in autonomous orsemi-autonomous vehicles.

“Input/Output (I/O) Device” generally refers to any device or collectionof devices coupled to a computing device that is configured to receiveinput and deliver the input to a processor, memory, or other part of thecomputing device and/or is controlled by the computing device to producean output. The I/O device can include physically separate input andoutput devices, or the input and output devices can be combined togetherto form a single physical unit. Such input devices of the I/O device caninclude keyboards, mice, trackballs, and touch sensitive pointingdevices such as touchpads, or touchscreens. Input devices also includeany sensor or sensor array for detecting environmental conditions suchas temperature, light, noise, vibration, humidity, and the like.Examples of output devices for the I/O device include, but are notlimited to, screens or monitors displaying graphical output, aprojecting device projecting a two-dimensional or three-dimensionalimage, or any kind of printer, plotter, or similar device producingeither two-dimensional or three-dimensional representations of theoutput fixed in any tangible medium (e.g. a laser printer printing onpaper, a lathe controlled to machine a piece of metal, or athree-dimensional printer producing an object). An output device mayalso produce intangible output such as, for example, data stored in adatabase, or electromagnetic energy transmitted through a medium orthrough free space such as audio produced by a speaker controlled by thecomputer, radio signals transmitted through free space, or pulses oflight passing through a fiber-optic cable.

“Inertial Measurement Unit” or “IMU” generally refers to a device thatmeasures and reports a body's specific force, angular rate, andsometimes the magnetic field surrounding the body. The IMU typically,but not always, includes one or more accelerometers and gyroscopes, andsometimes magnetometers when the surrounding magnetic fields aremeasured. IMUs are typically (but not always) self-contained systemsthat measure linear and angular motion usually with a triad ofgyroscopes and triad of accelerometers. An IMU can either be gimballedor strapdown, outputting the integrating quantities of angular velocityand acceleration in the sensor/body frame. They are commonly referred toin literature as the rate-integrating gyroscopes and accelerometers.IMUs typically can be used in a wide variety of circumstances such as tomaneuver vehicles, aircraft, and/or spacecraft as well as in cellphonesand virtual reality glasses. The accelerometers in IMUs can includemechanical and/or electronic type accelerometers, and the gyroscopes inIMUs can include mechanical and/or electronic type gyroscopes.

“Memory” generally refers to any storage system or device configured toretain data or information. Each memory may include one or more types ofsolid-state electronic memory, magnetic memory, or optical memory, justto name a few. Memory may use any suitable storage technology, orcombination of storage technologies, and may be volatile, nonvolatile,or a hybrid combination of volatile and nonvolatile varieties. By way ofnon-limiting example, each memory may include solid-state electronicRandom Access Memory (RAM), Sequentially Accessible Memory (SAM) (suchas the First-In, First-Out (FIFO) variety or the Last-In-First-Out(LIFO) variety), Programmable Read Only Memory (PROM), ElectronicallyProgrammable Read Only Memory (EPROM), or Electrically ErasableProgrammable Read Only Memory (EEPROM).

Memory can refer to Dynamic Random Access Memory (DRAM) or any variants,including static random access memory (SRAM), Burst SRAM or Synch BurstSRAM (BSRAM), Fast Page Mode DRAM (FPM DRAM), Enhanced DRAM (EDRAM),Extended Data Output RAM (EDO RAM), Extended Data Output DRAM (EDODRAM), Burst Extended Data Output DRAM (BEDO DRAM), Single Data RateSynchronous DRAM (SDR SDRAM), Double Data Rate SDRAM (DDR SDRAM), DirectRambus DRAM (DRDRAM), or Extreme Data Rate DRAM (XDR DRAM).

Memory can also refer to non-volatile storage technologies such asNon-Volatile Read Access memory (NVRAM), flash memory, non-volatileStatic RAM (nvSRAM), Ferroelectric RAM (FeRAM), Magnetoresistive RAM(MRAM), Phase-change memory (PRAM), Conductive-Bridging RAM (CBRAM),Silicon-Oxide-Nitride-Oxide-Silicon (SONOS), Resistive RAM (RRAM),Domain Wall Memory (DWM) or “Racetrack” memory, Nano-RAM (NRAM), orMillipede memory. Other nonvolatile types of memory include optical discmemory (such as a DVD or CD ROM), a magnetically encoded hard disc orhard disc platter, floppy disc, tape, or cartridge media. The concept ofa “memory” includes the use of any suitable storage technology or anycombination of storage technologies.

“Microphone” generally refers to a transducer that converts sound intoan electrical signal.

“Network” or “Computer Network” generally refers to a telecommunicationsnetwork that allows computers to exchange data. Computers can pass datato each other along data connections by transforming data into acollection of datagrams or packets. The connections between computersand the network may be established using either cables, optical fibers,or via electromagnetic transmissions such as for wireless networkdevices.

Computers coupled to a network may be referred to as “nodes” or as“hosts” and may originate, broadcast, route, or accept data from thenetwork. Nodes can include any computing device such as personalcomputers, phones, and servers as well as specialized computers thatoperate to maintain the flow of data across the network, referred to as“network devices”. Two nodes can be considered “networked together” whenone device is able to exchange information with another device, whetheror not they have a direct connection to each other.

Examples of wired network connections may include Digital SubscriberLines (DSL), coaxial cable lines, or optical fiber lines. The wirelessconnections may include BLUETOOTH®, Worldwide Interoperability forMicrowave Access (WiMAX), infrared channel or satellite band, or anywireless local area network (Wi-Fi) such as those implemented using theInstitute of Electrical and Electronics Engineers' (IEEE) 802.11standards (e.g. 802.11(a), 802.11(b), 802.11(g), or 802.11(n) to name afew). Wireless links may also include or use any cellular networkstandards used to communicate among mobile devices including 1G, 2G, 3G,or 4G. The network standards may qualify as 1G, 2G, etc. by fulfilling aspecification or standards such as the specifications maintained by theInternational Telecommunication Union (ITU). For example, a network maybe referred to as a “3G network” if it meets the criteria in theInternational Mobile Telecommunications-2000 (IMT-2000) specificationregardless of what it may otherwise be referred to. A network may bereferred to as a “4G network” if it meets the requirements of theInternational Mobile Telecommunications Advanced (IMTAdvanced)specification. Examples of cellular network or other wireless standardsinclude AMPS, GSM, GPRS, UMTS, LTE, LTE Advanced, Mobile WiMAX, andWiMAX-Advanced.

Cellular network standards may use various channel access methods suchas FDMA, TDMA, CDMA, or SDMA. Different types of data may be transmittedvia different links and standards, or the same types of data may betransmitted via different links and standards.

The geographical scope of the network may vary widely. Examples includea Body Area Network (BAN), a Personal Area Network (PAN), a Local-AreaNetwork (LAN), a Metropolitan Area Network (MAN), a Wide Area Network(WAN), or the Internet.

A network may have any suitable network topology defining the number anduse of the network connections. The network topology may be of anysuitable form and may include point-to-point, bus, star, ring, mesh, ortree. A network may be an overlay network which is virtual and isconfigured as one or more layers that use or “lay on top of” othernetworks.

A network may utilize different communication protocols or messagingtechniques including layers or stacks of protocols. Examples include theEthernet protocol, the internet protocol suite (TCP/IP), the ATM(Asynchronous Transfer Mode) technique, the SONET (Synchronous OpticalNetworking) protocol, or the SDE1 (Synchronous Digital Elierarchy)protocol. The TCP/IP internet protocol suite may include the applicationlayer, transport layer, internet layer (including, e.g., IPv6), or linklayer.

“Pallet” generally refers to a portable platform or other structure onwhich goods or items can be assembled, stacked, stored, packaged,handled, transported, and/or moved, such as with the aid of a forkliftor pallet jack, as a unit load. Typically, but not always, the pallet isrigid and forms a horizontal base upon which the items rest. Goods,shipping containers, and other items are often placed on a palletsecured with strapping, stretch wrap, and/or shrink wrap. Often, but notalways, the pallet is equipped with a superstructure. In one form, thepallet includes structures that support goods in a stable fashion whilebeing lifted by a forklift, pallet jack, front loader, and/or otherlifting devices. In particular, pallets typically include a top deckupon which items are stacked, a bottom deck that rests on the ground,and a spacer structure positioned between the top and bottom decks toreceive the forks of the forklift or pallet jack. However, the palletscan be configured differently. For example, the term pallet is used in abroader sense to include skids that have no bottom deck. One or morecomponents of the pallet, or even the entire pallet, can be integrallyformed together to form a single unit. By way of non-limiting examples,these pallets can include stringer, block, perimeter, skid, solid deck,multiple deck board, panel-deck, slave, double-deck (or face),single-way entry, two-way entry, four-way entry, flush, single-wing,double-wing, expendable, limited-use, multiple-use, returnable,recycled, heat treated, reversible, non-reversible, and/or warehousetype pallets.

“Processor” generally refers to one or more electronic componentsconfigured to operate as a single unit configured or programmed toprocess input to generate an output. Alternatively, when of amulti-component form, a processor may have one or more componentslocated remotely relative to the others. One or more components of eachprocessor may be of the electronic variety defining digital circuitry,analog circuitry, or both. In one example, each processor is of aconventional, integrated circuit microprocessor arrangement, such as oneor more PENTIUM, i3, i5 or i7 processors supplied by INTEL Corporationof 2200 Mission College Boulevard, Santa Clara, Calif. 95052, USA. Inanother example, the processor uses a Reduced Instruction Set Computing(RISC) architecture, such as an Advanced RISC Machine (ARM) typeprocessor developed and licensed by ARM Holdings of Cambridge, UnitedKingdom. In still yet other examples, the processor can include aCentral Processing Unit (CPU) and/or an Accelerated Processing Unit(APU), such as those using a K8, K10, Bulldozer, Bobcat, Jaguar, and Zenseries architectures, supplied by Advanced Micro Devices, Inc. (AMD) ofSanta Clara, Calif.

Another example of a processor is an Application-Specific IntegratedCircuit (ASIC). An ASIC is an Integrated Circuit (IC) customized toperform a specific series of logical operations for controlling thecomputer to perform specific tasks or functions. An ASIC is an exampleof a processor for a special purpose computer, rather than a processorconfigured for general-purpose use. An application-specific integratedcircuit generally is not reprogrammable to perform other functions andmay be programmed once when it is manufactured.

In another example, a processor may be of the “field programmable” type.Such processors may be programmed multiple times “in the field” toperform various specialized or general functions after they aremanufactured. A field-programmable processor may include aField-Programmable Gate Array (FPGA) in an integrated circuit in theprocessor. FPGA may be programmed to perform a specific series ofinstructions which may be retained in nonvolatile memory cells in theFPGA. The FPGA may be configured by a customer or a designer using aHardware Description Language (HDL). An FPGA may be reprogrammed usinganother computer to reconfigure the FPGA to implement a new set ofcommands or operating instructions. Such an operation may be executed inany suitable means such as by a firmware upgrade to the processorcircuitry.

Just as the concept of a computer is not limited to a single physicaldevice in a single location, so also the concept of a “processor” is notlimited to a single physical logic circuit or package of circuits butincludes one or more such circuits or circuit packages possiblycontained within or across multiple computers in numerous physicallocations. In a virtual computing environment, an unknown number ofphysical processors may be actively processing data, and the unknownnumber may automatically change over time as well.

The concept of a “processor” includes a device configured or programmedto make threshold comparisons, rules comparisons, calculations, orperform logical operations applying a rule to data yielding a logicalresult (e.g. “true” or “false”). Processing activities may occur inmultiple single processors on separate servers, on multiple processorsin a single server with separate processors, or on multiple processorsphysically remote from one another in separate computing devices.

“Safety Command” generally refers to a request for an action to beperformed that prevents imminent harm or damage to a human, a piece ofequipment, and/or a structure. The safety command can be communicated ina number of forms such as in verbal, written, symbolic, and/orelectronic forms. Some non-limiting examples, of safety commands includethe words “stop”, “halt”, “turn”, “back up”, “reverse”, or “slow down”to name just a few.

“Satellite navigation” generally refers to a system that uses satellitesto provide geo-spatial positioning data. In one example, the system mayinclude a receiver that interacts with satellites using electromagneticradiation. The timing of the transmission of the signal from thereceiver to the satellites allows calculation of the position of thereceiver using triangulation. Some of examples of satellite navigationsystems include global positioning systems such as GPS and GLONASS aswell as global positioning systems under development such as Galileo. Asatellite navigation system may also be a regional positioning systemsuch as BeiDou, NAVIC, and QZSS.

“Sensor” generally refers to an object whose purpose is to detect eventsand/or changes in the environment of the sensor, and then provide acorresponding output. Sensors include transducers that provide varioustypes of output, such as electrical and/or optical signals. By way ofnonlimiting examples, the sensors can include pressure sensors,ultrasonic sensors, humidity sensors, gas sensors, motion sensors,acceleration sensors, displacement sensors, force sensors, opticalsensors, and/or electromagnetic sensors. In some examples, the sensorsinclude barcode readers, RFID readers, and/or vision systems.

“Substantially” generally refers to the degree by which a quantitativerepresentation may vary from a stated reference without resulting in anessential change of the basic function of the subject matter at issue.The term “substantially” is utilized herein to represent the inherentdegree of uncertainty that may be attributed to any quantitativecomparison, value, measurement, and/or other representation.

“Transceiver” generally refers to a device that includes both atransmitter and a receiver that share common circuitry and/or a singlehousing. Transceivers are typically, but not always, designed totransmit and receive electronic signals, such as analog and/or digitalradio signals.

“Transmit” generally refers to causing something to be transferred,communicated, conveyed, relayed, dispatched, or forwarded. The conceptmay or may not include the act of conveying something from atransmitting entity to a receiving entity. For example, a transmissionmay be received without knowledge as to who or what transmitted it.Likewise the transmission may be sent with or without knowledge of whoor what is receiving it. To “transmit” may include, but is not limitedto, the act of sending or broadcasting electromagnetic energy at anysuitable frequency in the electromagnetic spectrum. Transmissions mayinclude digital signals which may define various types of binary datasuch as datagrams, packets and the like. A transmission may also includeanalog signals.

“Warehouse Management System” or “WMS” generally refers to a computersystem and associated software that allow organizations to control andadminister warehouse operations from the time goods or materials enter awarehouse, manufacturing plant, storage lot, and/or other inventoryfacility until the goods or materials move out of the facility.Operations managed by a WMS include, but are not limited to, inventorymanagement, picking processes and/or auditing.

The term “or” is inclusive, meaning “and/or”.

It should be noted that the singular forms “a,” “an,” “the,” and thelike as used in the description and/or the claims include the pluralforms unless expressly discussed otherwise. For example, if thespecification and/or claims refer to “a device” or “the device”, itincludes one or more of such devices.

It should be noted that directional terms, such as “up,” “down,” “top,”“bottom,” “lateral,” “longitudinal,” “radial,” “circumferential,”“horizontal,” “vertical,” etc., are used herein solely for theconvenience of the reader in order to aid in the reader's understandingof the illustrated embodiments, and it is not the intent that the use ofthese directional terms in any manner limit the described, illustrated,and/or claimed features to a specific direction and/or orientation.

While the invention has been illustrated and described in detail in thedrawings and foregoing description, the same is to be considered asillustrative and not restrictive in character, it being understood thatonly the preferred embodiment has been shown and described and that allchanges, equivalents, and modifications that come within the spirit ofthe inventions defined by the following claims are desired to beprotected. All publications, patents, and patent applications cited inthis specification are herein incorporated by reference as if eachindividual publication, patent, or patent application were specificallyand individually indicated to be incorporated by reference and set forthin its entirety herein.

Reference Numbers 100 AMU system 105 AMU 110 base station 115 lanyard120 WMS 125 network 205 microphone 210 controller 215 memory 220 GNCsystem 225 sensors 230 I/O device 235 transceiver 305 microphone 310processor 315 memory 320 I/O device 325 transceiver 405 microphone 410processor 415 memory 420 PDS 425 transceiver 430 ESS 435 strap slot 440strap 505 network interface 510 processor 515 memory 520 I/O device 605automated forklift 610 forks 615 arrows 620 individual 625 verbalinstruction 700 flowchart 705 stage 710 stage 715 stage 720 stage 725stage 730 stage 805 first storage location 810 first pallet 815 secondstorage location 820 second pallet 825 third storage location 830 thirdpallet 835 initial path 840 detour path 900 flowchart 905 stage 910stage 915 stage 920 stage 1005 inner safety zone 1010 outer safety zone1015 arrow 1020 arrow 1025 arrow 1100 flowchart 1105 stage 1110 stage1115 stage 1120 stage 1125 stage

What is claimed is:
 1. A system, comprising: an autonomous mobile unit(AMU) that is responsive to one or more voice commands.
 2. The system ofclaim 1, wherein the voice commands include safety control commands. 3.The system of claim 2, wherein the safety control commands areconfigured to stop the AMU.
 4. The system of claim 2, wherein the AMUhas a controller to process the voice commands locally to reducelatency.
 5. The system of claim 4, wherein the AMU is configured totransmit non-safety related control commands for remote processing. 6.The system of claim 4, wherein the controller includes a circuit boardintegrated with a microphone.
 7. The system of claim 1, wherein thevoice commands include requests for information.
 8. The system of claim1, wherein the voice commands include one or more system controlcommands for controlling functions of the AMU.
 9. The system of claim 8,wherein the system control commands control movement of the AMU.
 10. Thesystem of claim 1, wherein the AMU includes one or more microphones forreceiving the voice commands.
 11. The system of claim 10, wherein theAMU includes an Automated Guided Vehicle (AGV).
 12. The system of claim1, further comprising: a base station with a microphone for receivingthe voice commands to control the AMU.
 13. The system of claim 1,further comprising: a lanyard with a microphone for receiving the voicecommands to control the AMU.
 14. The system of claim 13, wherein thelanyard includes a tracking device for location tracking.
 15. The systemof claim 14, wherein the AMU is configured to perform a safety action inthe presence of the lanyard.
 16. The system of claim 1, wherein thevoice commands are configured to temporarily interrupt workflow of theAMU to perform a different task.
 17. A method, comprising: operating anautonomous mobile unit (AMU); and changing activity of the AMU inresponse to one or more voice commands.
 18. The method of claim 17,wherein the voice commands include one or more safety control commands.19. The method of claim 18, further comprising: stopping the AMU inresponse to the safety control commands.
 20. The method of claim 18,further comprising: processing the voice commands locally with acontroller of the AMU to reduce latency.
 21. The method of claim 20,further comprising: determining with the AMU that the voice commands arenon-safety related commands; and transmitting from the AMU thenon-safety related commands for remote processing.
 22. The method ofclaim 20, wherein the controller includes a circuit board integratedwith a microphone.
 23. The method of claim 17, wherein the voicecommands include one or more requests for information.
 24. The method ofclaim 17, wherein the voice commands include system control commands forcontrolling functions of the AMU.
 25. The method of claim 24, furthercomprising: controlling movement of the AMU based on the system controlcommands.
 26. The method of claim 17, further comprising: receiving thevoice commands with one or more microphones of the AMU.
 27. The methodof claim 26, wherein the AMU includes an Automated Guided Vehicle (AGV).28. The method of claim 17, further comprising: receiving the voicecommands with a microphone of a base station; and controlling the AMUwith the voice commands from the base station.
 29. The method of claim17, further comprising: receiving the voice commands with a microphoneof a lanyard; and controlling the AMU based on the voice commands fromthe lanyard.
 30. The method of claim 29, further comprising: trackinglocation of an individual with a Position Detection System (PDS) of thelanyard.
 31. The method of claim 30, further comprising: performing asafety action with the AMU in the presence of the lanyard.
 32. Themethod of claim 17, further comprising: performing a first task with theAMU; interrupting the first task in response to the voice commands toperform a second task; completing the second task with the AMU; andresuming the first task with the AMU.